Eliciting User Expectations for Data Behavior via Invariant Templates

People expect software that they use for everyday purposes to be dependable enough for their needs. Usually, they can tolerate some failures, provided they can notice and recover from problems. Many dependability enhancement techniques rely on failure detection. Detection requires a model of proper behavior, preferably in the form of specifications. However, the specifications of everyday software are often incomplete and imprecise. This research uses machine learning techniques to refine the model and accommodates the necessary human participation. We propose a template mechanism to bridge between user expectations and techniques output. The result is an analyzable model of proper behavior that may serve as a proxy for missing specifications. We use the model to detect semantic anomalies—data behavior that is outside the user’s expectations. We test our template mechanism on truck weigh-in-motion (WIM) data. A domain expert interacts with this mechanism to set up the model of proper behavior. We then analyze the usefulness of this model for anomaly detection and show it is useful. We also compare the model to existing documentation and show how this gave the expert insights about the WIM system. This research is supported by the National Science Foundation under Grant ITR-0086003, by the Sloan Software Industry Center at Carnegie Mellon University, and by the High Dependability Computing Program from NASA Ames cooperative agreement NCC-2-1298.